…reshold examples

…n, lib refactor map

…VT-237] (#226)

* feat(dtni): vllm_single PoC — typed configs + orch-driven VllmJob

Replace 4 byte-similar vllm_single wrappers with a single parametrized
suite, per-variant config + threshold dirs, a typed pydantic loader, and
a new orch-driven VllmJob whose container lifecycle is owned entirely by
ContainerOrchestrator (launch:true).

- cvs/lib/dtni/{verdict,config_loader}.py — 5 threshold kinds; pydantic v2
  models with extra=forbid; 3-pass placeholder substitution; model.remote=1
  raises NotImplementedError pointing at v1 resource_resolver.
- cvs/lib/inference/vllm_orch.py — standalone VllmJob driven by orch.exec.
  Drops dead self.port_no distributed branch, random_range_ration typo,
  globals.error_list indirection, silent-skip in verify_inference_results.
- cvs/tests/inference/vllm/{conftest,_shared,vllm_single}.py — orch fixture
  owns container lifetime; test_print_results_table moved to _shared.
- cvs/input/dtni/vllm_single/{4 variants}/{config,threshold}.json — all
  variants pinned to rocm/vllm-dev:nightly for the PoC; thresholds carry
  MI300X-realistic floors (~1/3 of MI355X totals) for the verification node.
- cvs/input/cluster_file/mi300x_g21u37.json — single-node MI300X cluster
  for verification on 10.245.135.13.
- Delete the 4 old per-model wrappers and mi355x_vllm_single.json.

Verification (offline gates): pytest --collect-only enumerates 9 parametric
cells + test_print_results_table; cvs list vllm_single discovers both test
functions; load_variant of a missing-models-dir variant resolves the
expected /models/{id} path; pydantic ValidationError fires on a
percentile_metrics typo. On-hardware verification is deferred: target node
lacks pre-fetched models, HF token, and benchmark server scripts.

Legacy cvs/lib/inference/{base,vllm,inference_max}.py untouched; other
suites (sglang, inferencemax, pytorch_xdit, megatron, jax) unaffected.

* fix(dtni): cluster file uses devbox-correct key path

The devbox /data/atnair is /data/atnair (not /home/atnair), and the node
10.245.135.13 authenticates with id_ed25519 (not id_rsa). Update the
verification cluster file so the orch fixture authenticates first try.
Confirmed via a lifecycle smoke that brought up an alpine container on
the node and tore it down at the right boundaries.

* fix(dtni): remediate vllm_single lifecycle review findings

- is_ready: grep in-container instead of cat-ing the whole server log
- thresholds: fail at load if sweep cells lack a threshold entry; hard-error
  (not silent skip) on per-cell verdict miss
- HTML report: per-test timing rows with explicit units (no cross-row leak)
- client failure: treat only a nonzero failed-request count as failure
- pin HF cache to the mounted models dir; shlex-quote shell interpolation
- raise server readiness budget to 60min for remote model pulls
- remove legacy cvs.lib.inference.vllm.VllmJob (no remaining importers)

* refactor(dtni): collapse vllm_single to single W1 config + generic cluster file

- Replace 4 model variants under input/dtni/vllm_single/ with one W1 config
  (Llama 3.1 70B FP8-KV, TP=8) at input/config_file/inference/vllm_single/.
- Rename cluster file to mi300x_vllm_single.json with <changeme> placeholders
  so it is generic/shareable rather than node-specific.
- config_loader: add enforce_thresholds gate (record-only scaffolds),
  glob threshold sibling, drop enumerate_variants, note generalization seam.
- Move pytest_generate_tests into vllm_single test module; drop aa/ab/zz
  lifecycle-ordering prefixes from test names.
- Drop unused imports / apply ruff formatting across touched lib + test files.

* fix(dtni): address vllm_single PR review findings

- config_loader: drop dead BenchmarkParams class + unused benchmark_params
  field (and the matching key in the w1 config)
- config_loader: raise FileNotFoundError/ValueError instead of AssertionError
  in load_variant (AssertionError is stripped under python -O)
- config_loader: collapse _resolve_cluster_mapping; clarify the container
  runtime docstring and the intentional model_validator ordering
- vllm_orch: build the bench client command as a shlex.quote-d arg list so a
  model id or path containing a space or $ cannot break the inner bash layer

* fix(dtni): address second-round vllm_single review

- conftest: scope inf_res_dict per-module to match sibling fixtures and
  avoid cross-module result-table bleed
- test_model_fetch: split the offline/pre-staged path from the download
  poll loop; presence-check with retries so a slow mount that reads 0 on
  the first du does not false-fail a model that is present
- start_server: shlex.quote scripts_dir/server_script/server_log, matching
  the per-path quoting used elsewhere in the file
- test_teardown: set lifecycle.torn_down only after verifying the container
  is gone so the orch finalizer retries an incomplete teardown
- _clone_bench_serving: document why the bench_serving URL is a hardcoded
  calibration fork (not stock vLLM, unpinned HEAD), kept for legacy parity

…#227)

* feat(dtni): move vllm bench client to stock vllm bench serve

Drop the kimbochen/bench_serving fork clone in VllmJob.run_client and
invoke the in-image stock "vllm bench serve" CLI instead. The fork was
cloned at unpinned HEAD; pinning the client to the run image tag gives
Spec 1 a stable artifact contract to parse.

- run_client: remove _clone_bench_serving call; head tokens become
  vllm/bench/serve; drop the now-dead cd /app in client_cmd
- remove _clone_bench_serving and its (false) calibration comment
- drop Params.bench_serv_script (extra=forbid) and the matching key in
  the vllm_single config; rename client_log to client.log

base.py / inferencemax still use the fork (separate workload, untouched).
Source-only change; no metrics added (Spec 1). enforce_thresholds=false.

* fix(dtni): robust client completion + launch-failure detection for stock bench

Harden wait_client_complete after the move to stock vllm bench serve:

- COMPLETION_RE: key off the unconditional "Serving Benchmark Result"
  banner instead of the "End-to-end Latency" metric header. Stock prints
  metric headers only when the metric is in --percentile-metrics, so a
  config omitting e2el would never be detected as complete and would spin
  to the poll cap (~90 min) on an otherwise-successful run.
- add CLIENT_LAUNCH_FAIL_RE: a CLI launch failure (bad/renamed flag,
  missing bench subcommand, vllm not on PATH) exits before any summary and
  is neither a Python traceback nor a Failed-requests line, so it too would
  hang the poll cap. Treat it as a hard failure, like a crash.
- drop dead export RESULT_FILENAME=results: consumed only by the removed
  fork client; stock takes the name via --result-filename.

Hnimrama/inferencemax uplift

Refactors InferenceMax for the DTNI pytest layout (inferencemax_single): ContainerOrchestrator-based
conftest, suite/threshold JSON loading, benchmark model selection, and tighter server/client
lifecycle handling against current InferenceX upstream.

Benchmarking: stop cloning third-party bench_serving; resolve benchmark_serving.py from the
installed vllm package (BENCH_SCRIPT) for InferenceMax and vLLM single paths. Host-mounted server
entrypoints live under cvs.lib.dtni.vllm_benchmark_scripts (vllm_serve_mi300x.sh); samples and docs
use <changeme> container placeholders, legacy benchmark_script_repo called out as ignored, and
volume_dict guidance avoids duplicate Docker :/workspace mounts.

vLLM single (vllm_orch): align with dev/dtni completion and client-failure detection while keeping
python3 "$BENCH_SCRIPT" invocation.

Misc: optional run_plugin --log-file; sglang_disagg total_generated_tokens key; log redaction and
small review fixes from PR feedback.

Test with cvs run inferencemax_single (cluster + suite JSON, HF token) and spot-check vllm_single
if configs touch shared modules.

…t" (#228)

This reverts commit 5798f4e.

ContainerOrchestrator.setup_sshd() ran a fixed command list ending in
`/usr/sbin/sshd -p2224` and asserted every step succeeded, for every
container run regardless of node count. The orch pytest fixture calls it
unconditionally, so a single-node run on a minimal image with no
/usr/sbin/sshd failed the whole fixture with a generic "SSH setup command
failed" message and never ran the workload.

The in-container sshd exists only so MPI (mpirun's plm_rsh_args -p 2224)
can reach peer ranks on other nodes. A single-node run execs directly via
docker exec and never distributes over MPI, so the sshd setup is dead
weight there. Guard setup_sshd() to return True early when len(self.hosts)
<= 1, after the container_id precondition. The host count lives on the
orchestrator, so the decision belongs there; multinode runs are unchanged.

…llm result path (#233)

* refactor(lib): rename dtni -> utils and split out generic config machinery

Rename cvs/lib/dtni to cvs/lib/utils ("utils" says what it is: pure
functions any lib can call; "dtni" was a leftover project codename).
verdict.py moves unchanged.

config_loader.py is trimmed to the framework-agnostic half: the
paths/model/image/container schema, the 3-pass placeholder substitution,
the enforce_thresholds gate on a new BaseVariantConfig, and a
substitute_config() helper (file read + substitution + sibling-threshold
discovery). The inference-only schema moves to a sibling module in a
later commit.

* refactor(inference): move vllm_parsing into cvs/lib/inference/utils

The client.* metric parser (to_client_metrics + CLIENT_METRICS surface)
is inference-specific and should not sit in the shared utils dir. Move it
under a new cvs/lib/inference/utils package. Content unchanged.

* feat(inference): inference config schema with named-combo sweep selector

The inference half of the old config_loader: GoodputSlo, SeqCombo, Sweep,
Params, Roles, and VariantConfig(BaseVariantConfig) with cell_key and the
threshold-coverage check. load_variant() delegates the file read and
placeholder substitution to the generic substitute_config().

Replaces the sequence_combinations x concurrency_levels cartesian with a
named-combo + explicit runs[] selector: each run is a {combo, concurrency}
pair, so the config enumerates exactly the cells to run (no NxM explosion).
A model_validator rejects duplicate combo names and runs referencing an
unknown combo at load time.

* feat(inference): self-contained server cmd + artifact-based result parsing

build_server_cmd/start_server assemble a `vllm serve` arg list in Python
(mirroring run_client) instead of cloning and running an external .sh, so
a run needs no hand-staged script. --max-model-len is derived per cell
from isl/osl/random_range_ratio so any sweep change stays self-consistent.

parse_results reads the stock extensionless `results` JSON artifact that
`vllm bench serve` writes to --result-dir and delegates namespacing +
derived-metric math to vllm_parsing.to_client_metrics, replacing the
brittle console-log regex table. Missing/empty/unparseable artifacts
hard-fail the cell rather than recording a silently-green empty row.

Adds the optional --goodput SLO gate (per-cell, omitted when no SLO) and
threads goodput_slo through run().

* feat(suite): per-metric result rows, Value/Unit columns, sweep selector

pytest_generate_tests now drives parametrization from the named-combo +
runs[] selector instead of the cartesian. test_vllm_inference only runs
the benchmark and stashes results; the verdict moves into a new
test_metric (one pytest test = one HTML row per metric per cell), with
inline Value/Unit columns added via pytest_html hooks in conftest.

test_setup_sshd gates its 2224 probe on len(orch.hosts) > 1, mirroring
the single-node orchestrator guard: single-node runs skip the in-container
sshd (it exists only for inter-node MPI) and must not probe for it.

Import paths follow the dtni -> utils / inference.utils moves.

* chore(config): rename vllm_single config pair, adopt selector, drop cluster file

Rename the config/threshold pair to {model}_{precision}_{config|threshold}
.json and convert the sweep to the named-combo + runs[] selector. Pin the
image to rocm/vllm-dev:nightly (the previously pinned :nightly-sshd tag
does not exist on Docker Hub, and single-node runs skip in-container sshd).

Delete cvs/input/cluster_file/mi300x_vllm_single.json: a cluster file only
needs node IP + user/key/orchestrator; the variant config supplies the
container block, so the bespoke per-suite cluster file is redundant.

* test(inference): unit tests for parser, sweep selector, and verdict guards

Cover to_client_metrics purity + derived metrics, the named-combo/runs
sweep selector (expansion, unknown-combo and duplicate-name rejection),
the run_client goodput/metric-percentiles flags, table-cell rendering, and
the verdict None-guards. Adds JSON fixtures for the stock results artifact.

* docs: suite-authoring guide + AGENTS.md for shared and inference helpers

Add a human reference guide (plans/building-a-cvs-test-suite.md) that walks
the six-layer suite architecture using vllm_single as the worked example:
the generic <-> framework config seam, the named-combo + runs[] sweep
selector, the self-contained Python-built server cmd, lifecycle-as-tests,
and a checklist for authoring a new inference or training suite.

Add per-package AGENTS.md docs naming the public entry points, the seam,
and the non-obvious gotchas:
  - cvs/lib/utils: substitute_config / BaseVariantConfig / evaluate_all,
    the 3-pass placeholder order, sibling-glob threshold discovery,
    parent-first validator ordering.
  - cvs/lib/inference/utils: load_variant / to_client_metrics / CLIENT_METRICS,
    the cell_key single-source-of-truth, the coverage check that prevents a
    silent green, and the validators mirrored in pytest_generate_tests.

* docs: expand suite guide with lib restructure, drop redundant section rules

Document the dtni -> utils rename and the shared (cvs/lib/utils) vs
domain-specific (cvs/lib/inference/utils) split: what lives where, the rule
for placing a new helper, and the directory map. Note training is not yet
ported and this guide is the blueprint for that port.

Remove the manual --- horizontal rules between sections: heading levels
already render their own bottom border, so the extra rules produced a
double-underline. Minor prose/format cleanups.

* docs: demote headings so GitHub stops underlining sections

* removing old plan

* fix(config): re-key W1 threshold to the swept CONC=16 cell

The threshold file carried placeholder CONC=64/128/256 entries while the
sweep's only run is concurrency 16, so cell_key() matched no threshold.
The mismatch was masked by enforce_thresholds=false (warned, not raised)
and would have failed load the instant enforcement was flipped on. Re-key
to the single CONC=16 cell the runs selector actually enumerates.

* fix(inference): drop dead server-env exports from build_server_cmd

MODEL/ISL/OSL/MAX_MODEL_LEN/RANDOM_RANGE_RATIO/TP/CONC/PORT were exported
into /tmp/server_env_script.sh but read by nothing after the .sh->Python
server command refactor -- both _server_argv and run_client pass these as
explicit flags. Keep only the env the vllm process actually consumes
(HF token, HF cache pin, AITER flags). Also drops the second
_derive_max_model_len call that fed the dead MAX_MODEL_LEN export.

* fix(inference): move --kv-cache-dtype out of the driver into config

_server_argv hard-coded --kv-cache-dtype fp8, baking a per-model property
into the shared orchestrator -- a non-fp8-KV model dropped into the suite
would be served wrong with no config recourse (extra_serve_args can only
add, so an override would pass the flag twice). Declare it in the W1
config's roles.server.extra_serve_args instead; the driver stays
model-agnostic and 'new model = new config' holds.

* refactor(inference): share the sweep-selector validator across load and collection

pytest_generate_tests hand-reimplemented the duplicate-name and
unknown-run.combo checks that Sweep._check_runs_reference_known_combos
already enforces, with divergent semantics (first-failure raise vs
all-at-once). Extract validate_sweep_selector() as the single home and
call it from both the typed validator (load time) and the collection-time
raw-JSON path so the rule can't drift.

* fix(inference): key the per-cell out_dir by isl/osl/conc

out_dir was fixed per job, so a multi-cell sweep would overwrite each
cell's `results` and client.log, and parse_results could cat a prior
cell's stale artifact if the current cell's client failed to write one.
Key it by cell. Latent today (the shipped sweep has one cell).

* refactor(inference): normalize goodput_slo to dict-only

run_client accepted goodput_slo as either a dict (.get) or an object
(getattr) via a per-key hasattr branch, but the only production caller
passes a raw dict -- the object path existed solely for a unit test, and
the dual path meant the typed GoodputSlo's validation never reached the
command builder. Consume the dict only and drop the object-form test.

* test(inference): drop unused _fake_variant parameter

goodput_slo_unused was never read (goodput is threaded through _make_job).

* chore(inference): placeholder personal/image refs in example config

The committed vllm_single example config carried a personal hf-token path
and a concrete image tag (duplicated in image.tag and container.image).
Replace all three with <changeme> so the file is a template a new user must
fill in -- it still loads (collection works) and only fails at run time when
an unedited value is read, which is the intended signal.

* refactor(inference): server serve_args as a {flag: value} map

The per-model server knobs were a flat [flag, value, flag, value] list
(roles.server.extra_serve_args), which reads poorly. Replace with a
roles.server.serve_args {flag: value} map (flag without the leading --):
a scalar renders --flag value, True a bare --flag, a list the flag once per
element -- so it stays readable while still covering vllm bare/repeatable
flags. _server_argv flattens the map via a new _flatten_serve_args helper;
the derived flags (tp/max-model-len/port) stay code-built.

Also repoints a stale unit test that asserted MAX_MODEL_LEN in the env
script (it moved to the --max-model-len flag in an earlier commit) to assert
against the server argv instead.

* feat(verdict): add unit-agnostic max threshold kind

The only ceiling kind was max_ms, whose message hard-codes ms. A count
metric like client.failed needs an upper bound without the unit lie; add
a plain max with the same comparison and an honest message.

* feat(inference): enforce a declared gated-metric SLO contract

Previously only cell-presence was validated: a cell could exist while a
given metric had no spec, and test_metric (spec is None -> return) would
silently report a green record-only row even under enforce_thresholds=true.
A new perf metric was thus unvalidated by default.

Declare GATED_METRICS beside CLIENT_METRICS -- the perf+health subset that
must assert (throughput, mean+p99 latency, success_rate/failed) -- and
extend _check_thresholds_cover_sweep to require a spec for every gated
metric in every present cell, reusing the same enforce-vs-warn path. A new
metric is record-only until added to the set; once gated, the loader forces
a spec in every cell before the suite can run green. Inputs, totals, and
derived diagnostics stay record-only by design.

* fix(inference): single image source on container.image

The image was declared twice: top-level image.tag (live -- conftest copied
it onto the container block) and container.image (dead -- overwritten by that
copy). The duplicate forced a top-level image block whose remote field was
unused and whose tag silently shadowed container.image.

Make container.image the single source: drop the top-level ImageSpec block
from the generic BaseVariantConfig, drop the conftest overwrite so the merged
container.image is used as-is, and remove the now-schemaless image block from
the example config.

* feat(inference): gate the full latency distribution

Expand GATED_METRICS from the mean+p99 subset to every emitted latency
quantile (mean/median/p90/p95/p99) for ttft, tpot, itl, and e2el -- itl
omits p90 as CLIENT_METRICS has no producer for it. Throughput and
success_rate/failed health are unchanged. Inputs, totals, secondary
throughputs, and derived diagnostics stay record-only.

The example threshold file gains a placeholder spec for each newly gated
metric (23 total) so the loader gated-coverage check passes.

* docs: reflect image collapse, serve_args rename, max kind, GATED_METRICS

- utils/AGENTS.md: drop top-level image (now container.image); add max verdict kind
- inference/utils/AGENTS.md: document GATED_METRICS contract + dual-axis coverage check
- building-a-cvs-test-suite.md: container.image, serve_args map, max kind, GATED_METRICS
- dtni-dev-guide.md: SUPERSEDED banner pointing to building guide + AGENTS.md

* refactor(inference): rename vllm_orch → vllm_single throughout

The module name vllm_orch.py implied disaggregated orchestration; this is a
single-node suite. Align the module name with the suite file and suite name.
Update all import sites, AGENTS.md prose, and the plan doc.

* fix(config): drop ModelSpec.precision, make threshold_json an explicit field

ModelSpec.precision was an unvalidated free-text field with no downstream use;
the kv-cache-dtype flag belongs in serve_args. Removing it prevents configs
silently carrying a stale or misleading label.

Replace the sibling-glob threshold discovery (glob('*threshold.json') next to
the config) with an explicit threshold_json field on BaseVariantConfig. The
glob was fragile: ambiguous when multiple threshold files coexist, and
invisible in the config spec. An explicit absolute path is transparent,
repo-portable, and validated as part of the schema.

Update the example config to add threshold_json: "<changeme>" and drop
model.precision.

* test(config): unit tests for ModelSpec, BaseVariantConfig, substitute_config

Cover the contracts changed by the precision-removal and threshold_json-explicit
commits: ModelSpec forbids precision and extra keys; BaseVariantConfig requires
threshold_json; substitute_config reads the threshold via raw['threshold_json']
as a literal absolute path, not by globbing the sibling directory; a sibling
*threshold.json must NOT be auto-discovered (regression guard for old behavior);
comment keys are stripped. No hardware; pure filesystem via tempfile.

* refactor(inference): O(n) duplicate detection in validate_sweep_selector

Replace list.count() inside the loop (O(n²)) with Counter: one pass to build
the frequency map, one comprehension to collect duplicates. Suggested in review.

* test(inference): address review — setUpClass, if __name__ at end of file

Convert per-test module loads to setUpClass so each heavy import runs once
per class, not once per test method: TestTableCellRendering._cell() (loads
_shared.py + stubs tabulate), TestKeyConsistency._producer_keys() (runs
parse_results), TestMetricTests.setUp() (_load_vllm_single). Suggested in
review for TestTableCellRendering; applied the same fix to the other two
classes that had the identical problem.

Move `if __name__ == "__main__": unittest.main()` from mid-file (line 282)
to the very end. Test classes defined after the guard were still discovered
by both pytest and python -m unittest (Python parses the full file first),
but the placement looked like dead code. Flagged in review.

* test(inference): address review + expand coverage for config loader models

Review items: remove dead `vc = _variant(sw)` assignment that was never read;
move `import warnings` from inline to top-level imports.

New test classes covering contracts changed in this PR: TestModelSpecNoPrecision
(extra field rejected), TestThresholdJsonField (required field, constructs with
it), TestCellCoverageAxis (missing cell and extra threshold-key axes, warn/raise
modes), TestExpectedCellsBoundaries (empty runs, unreferenced combo),
TestGoodputSlo (construction, missing fields, forbid extra keys, optional on
SeqCombo), TestSeqComboForbid (required fields, extra keys). All no-hardware.

* feat(config): expand w1 llama31_70b_fp8kv sweep to 5 ISL/OSL cells at conc=16

Replace the single ISL=128/OSL=2048 placeholder cell with the full
ISL/OSL matrix requested in review:

  ISL=1024 / OSL=1024
  ISL=8192 / OSL=1024
  ISL=1K   / OSL=8192
  ISL=1K   / OSL=4096
  ISL=5000 / OSL=1024

All five cells run at concurrency=16, TP=8, with record-only placeholder
thresholds (enforce_thresholds=false). threshold.json carries a spec for
every GATED_METRICS member per cell so the loader coverage check passes.

* fix(config): use literal ISL/OSL values (1000/8000/4000 not 1024/8192/4096)

* style(inference): collapse multi-line string concat in launch cmd

Co-Authored-By: Claude <noreply@anthropic.com>

* adding documentation

* renaming config and threshold files

---------

Co-authored-by: Claude <noreply@anthropic.com>

…e inference

Introduces vllm_distributed, a new CVS inference validation framework for
2-node MI300X clusters running vLLM with tensor parallelism (TP=8) and
pipeline parallelism (PP=2) across 16 GPUs total via the multiprocessing
distributed executor backend.

New files:
  cvs/lib/inference/vllm_distributed.py          VllmDistributedJob class:
    - build_server_cmd applies 5 in-container patches per run to fix upstream
      vLLM bugs in the rocm/ufb-private nightlies image:
        Patch 0:  delete stale multiproc_executor.pyc and core.pyc
        Patch 0b: replace assert in multiproc_executor.py:collective_rpc
                  (rpc_broadcast_mq is None on PP follower nodes); return
                  safe default instead of crashing
        Patch 1:  guard _initialize_kv_caches() for follower nodes; use
                  dummy KVCacheConfig(num_blocks=1) to skip collective_rpc
        Patch 2:  stub Scheduler() with _F on follower nodes to skip
                  KVCacheManager/HybridKVCacheCoordinator assert
        Patch 3:  fix get_supported_tasks() to return ("generate",) for
                  follower nodes (SupportedTask is Literal, not Enum)
    - is_ready() / wait_ready(): per-poll readiness with fatal-log detection
    - run_client(): bench serve head-only via exec_on_head
    - postcheck(): validates server log, client log, result file
    - collect_logs(): zips node logs and HTML artifacts
  cvs/lib/inference/utils/vllm_distributed_config_loader.py  config schema
  cvs/lib/inference/unittests/test_vllm_distributed.py        52 unit tests
  cvs/tests/inference/vllm_distributed/                       pytest suite
  cvs/input/config_file/inference/vllm_distributed/           config + thresholds

Modified files:
  cvs/core/orchestrators/container.py    openssh-server fallback install for
                                         images without sshd; per-cmd timeout
  cvs/lib/inference_lib.py               register vllm_distributed framework
  cvs/lib/inference/unittests/test_vllm_orch_parse.py  fix threshold JSON path

Validated on 10.245.135.15 (g21u43, head) + 10.245.135.115 (h16u07, worker)
with amd/Llama-3.1-70B-Instruct-FP8-KV, ISL=1000 OSL=1000 concurrency=16.

Signed-off-by: Atul Nair <Atul.Nair@amd.com>

Signed-off-by: Atul Nair <Atul.Nair@amd.com>